In recent years, it has been strongly required for imaging devices to have a small size in addition to a larger number of pixels. In many cases, the reduction in size is obstructed by the size and focal length of an optical lens and the size of an imaging device.
In general, when the wavelength of light varies, the refractive index of a material varies, and accordingly the focal length also varies. Thus, it is impossible to form a subject image including information of light of all wavelengths on an imaging plane using a single lens. Therefore, in an optical system of usual imaging devices, a plurality of optical lenses are arranged along an optical axis direction so that images of red wavelength light, green wavelength light, and blue wavelength light are formed on the same imaging plane. This makes the optical length longer, resulting in a thicker imaging device.
Therefore, as a technique effective for a reduction in size of imaging devices, particularly for a reduction in thickness, a compound-eye imaging device has been proposed in which a plurality of single lenses having a short focal length are arranged substantially in the same plane (see, Patent Document 1, for example). In a compound-eye color imaging device, a lens for forming an image of blue wavelength light, a lens for forming an image of green wavelength light, and a lens for forming an image of red wavelength light are arranged in the same plane, and imaging planes of imaging devices are arranged on optical axes of the respective lenses. Since the wavelength region of light handled by each lens is limited narrowly, it becomes possible to form a plurality of subject images on the plurality of imaging planes arranged in the same plane using the plurality of single lenses by making the focal lengths of the respective lenses equal. Therefore, the thickness of the imaging device can be reduced considerably.
FIG. 16 is a perspective view showing an exemplary compound-eye imaging device. Numeral 500 denotes a lens array including four lenses 501a, 501b, 501c, and 501d that are formed integrally. The lens 501a is for forming an image of red wavelength light, and forms a subject image on an imaging region 502a of an imaging device where a red wavelength separation filter (color filter) is attached to an imaging plane. The lenses 501b and 501d are for forming an image of green wavelength light, and form subject images on imaging regions 502b and 502d of imaging devices where a green wavelength separation filter (color filter) is attached to imaging planes. The lens 501c is for forming an image of blue wavelength light, and forms a subject image on an imaging region 502c of an imaging device where a blue wavelength separation filter (color filter) is attached to an imaging plane. The imaging devices convert the light intensity of the subject images formed on the respective imaging regions 502a to 502d into image data for output. These image data are superposed and synthesized, thereby obtaining a color image. It should be noted that the number of the lenses is not limited to four.
Although a compound-eye imaging device can achieve a reduced thickness as described above, it has a problem of a poorer resolution compared with a usual single-eye imaging device including color filters in a Bayer arrangement. The following is a description of the reason for this.
The single-eye imaging device is provided with color filters in a Bayer arrangement on incident planes of a large number of pixels so that each of the pixels in an imaging device can take out predetermined color information. In other words, the color filters transmitting green light are arranged in a checkered pattern so as to correspond to the arrangement of the large number of pixels arranged in a matrix, and the color filters transmitting red light and those transmitting blue light are arranged alternately in the rest of the pixels. The arrangement of the color filters in this manner generally is called a Bayer arrangement. Each of the pixels in the imaging device only outputs color information of the wavelength region of light that is transmitted by the color filter, and does not output color information of the wavelength region of light that is not transmitted thereby. However, since pieces of image information of three colors are known to be correlated in a local region of an image (e.g., Non-Patent Document 1), the green image information can be estimated from the red or blue image information, for example. Utilizing such characteristics, the image information of a missing color is interpolated. Accordingly, it is possible to obtain a color image having a resolution corresponding to pixels as numerous as the number of effective pixels in the imaging device. For example, in the case of using an imaging device with 1,000,000 effective pixels, 500,000 pixels detect green image information, 250,000 pixels detect blue image information, and 250,000 pixels detect red image information. However, by the above-described interpolation, it is possible to obtain image information having a resolution corresponding to 1,000,000 pixels for each of red, green and blue.
On the other hand, in the compound-eye imaging device, since each of the imaging regions of the imaging devices corresponding to the respective colors acquires any of red, green, and blue image information, a color image having a resolution corresponding to pixels as numerous as the pixels in that imaging region is achieved. For example, in order for each of the imaging regions corresponding to red, green, and blue to have 250,000 pixels, the imaging device needs to have 1,000,000 pixels in total, but the resolution of a color image obtained by superposition corresponds to 250,000 pixels.
As a method for improving the image resolution, there is a technology called “pixel shifting” in which a plurality of images whose relative positional relationships between a subject image and pixels in an imaging device are shifted from each other are acquired by shifting relative positions of a lens and a subject in time series using an actuator, and then synthesized so as to achieve a high-resolution image (e.g., Patent Document 2). In the pixel shifting technology, the optimal amount of shifting is determined by the direction of shifting and the number of images to be acquired. For example, in the case of synthesizing two images, when the relative positional relationship between the subject image and the pixels is shifted by one-half the arrangement pitch of the pixels (in the following, referred to as the “pixel pitch”) between the two images, it is possible to obtain the highest resolution image. The pixel shifting technology is applicable as long as it is possible to acquire a plurality of images whose relative positional relationships between the subject image formed by the lens and the pixels in the imaging device are shifted from each other, and can be applied also to the compound-eye imaging device. However, although the pixel shifting technology in which the relative positions of the lens and the subject are shifted in time series is effective for a static subject image, in the case of a non-static subject image, it is difficult to obtain a high-resolution image since the time series shifting of the relative positions of the lens and the subject delays the timing for capturing an image.
Further, Patent Document 3, for example, proposes a method for obtaining a high-resolution synthesized image in a compound-eye imaging device using a plurality of lenses to form a plurality of subject images on a plurality of imaging regions by positioning optical axes of the plurality of lenses with respect to pixels in an imaging device so that the pixel shifting is achieved such that the images of a subject at a predetermined distance are formed so as to be shifted by a predetermined amount in a direction connecting the optical axes of the lenses. Even in the case of a non-static subject image, this method can achieve a high-resolution image as long as the distance between the subject and the imaging device is fixed.    Patent Document 1: JP 2002-204462 A    Patent Document 2: JP 10 (1998)-304235 A    Patent Document 3: JP 2002-209226 A    Non-Patent Document 1: Hiroaki KOTERA and two others, “Representation of Full Color Image from a Single Color Image Using Color Correlation” Proceedings of 1988 Annual Conference of The Institute of Image Electronics Engineers of Japan 20, pp. 83-86 (1988)